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Abstract 

We consider the classical fc-rneans clustering problem in the setting bi-criteria ap¬ 
proximation, in which an algoithm is allowed to output / 3k > k clusters, and must 
produce a clustering with cost at most a times the to the cost of the optimal set of 
k clusters. We argue that this approach is natural in many settings, for which the 
exact number of clusters is a priori unknown, or unimportant up to a constant factor. 
We give new bi-criteria approximation algorithms, based on linear programming and 
local search, respectively, which attain a guarantee a(/3) depending on the number 
/ 3k of clusters that may be opened. Our gurantee a(/3) is always at most 9 + e and 
improves rapidly with /3 (for example: a(2) < 2.59, and a(3) < 1.4). Moreover, our 
algorithms have only polynomial dependence on the dimension of the input data, and 
so are applicable in high-dimensional settings. 


1 Introduction 

The /c-means clustering problem is one of the most popular models for unsupervised Machine 
Learning. The problem is formally defined as follows. 

Definition 1. In the /e-means problem, we are given a set X of n points x ±,... ,x n in W 3 
and an integer parameter k > 1. Our goal is to partition X into k clusters S\,... ,Sk and 
assign each cluster a center a* so as to minimize the cost XAi YlxjeSi 11A — a J| 2 - 

The most common heuristic for fc-means is Lloyd’s algorithm introduced in 1957 H3 
HE]- Llloyd’s algorithm starts with some initial solution and then iteratively improves it by 
alternating two steps: at the first step, the algorithm picks the optimal clustering for the 
current set of centers; at the second step, the algorithm picks the optimal set of centers for 
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the current clustering. While we know that the algorithm performs well on wcll-clusterable 
data [2T] it performs arbitrarily badly on general instances. There exist many variants of 
this algorithm and many heuristics for picking the initial solution. Unfortunately, none of 
them give a constant (not depending on k) factor approximation. One of the most popular 
ones is the k- means-H—f- algorithm that has an 0(log /^-approximation factor |5j. 

The general fc-means clustering problem has recently been shown to be APX-hard, ruling 
out a PTAS in the general case [7J. However, a variety of PTASes exist for special cases of 
the problem. Inaba, Katoh, and Imai [12] gave a (1 + e) - approximation algorithm for the 
case in which the number of clusters, k, and the dimension of the space, d, are fixed. Since 
then many more PTASes were proposed for other special cases. Additionally, there are many 
results showing the NP-hardness for several special cases mmm- 

The best constant factor approximation algorithm for the general case of the problem 
was proposed by [IB] a decade ago. Their algorithm gives 9 + e factor approximation. 
Using the connection with /e-median problem j4] designed an alternative constant factor 
approximation algorithm. [2] showed that running the /c-means H—K algorithm for more steps 
gives an a — 4 + e factor approximation by opening [16 (fc + Vk)] centers, and also showed 
how to modify the resulting solution to obtain a set of k centers attaining an 0(1) factor 
guarantee. 

In most practical applications the target number k of clusters is not fixed in advance, 
rather we would like to find a number k that provides a well-cluster able solution. Here, 
we show how to substantially improve the approximation factor by slightly violating the 
constraint on the number of clusters. We present bi-criteria approximation algorithms for 
the general case of the problem. A (/3, a) bi-criteria approximation algorithm finds a solution 
with /3k clusters, whose cost is at most a times the optimal cost of a solution using k clusters. 
In contrast to the approach of [2], our algorithms find an approximate solution for every 
(3 > 1. Our approximation is always at most 9, and decreases rapidly with (3. In particular, 
we obtain a 4-approximation by opening only 1.65 k centers, improving over previous results 
[2] by a factor of nearly 10, and obtain improved approximation factors a((3) as f3 continues 
to grow. For example, a(1.3) < 6.45, a(1.5) < 4.8; a( 2) < 2.59, and a(3) < 1.4. In general, 
we argue that in many applications the number of clusters is not important as long as it 
approximately equals k. For these applications we can obtain an approximation factor very 
close to 1. 

We give three bi-criteria algorithms—two based on linear programming and one based on 
local search. We show the algorithms’ approximation factors as a function of (3 in Figured] 
Note that our linear programming algorithm attains a better approximation a for large /3, 
while the local search algorithm is better for /3 near 1. 

Both of our algorithms are based on a reduction from the /e-means problem, in which 
cluster centers may be placed at any point in M p , to the following /e-median problem, in which 
we are restricted to a given, discrete set of candidate cluster centers with specified distances 
from each point. As part of reduction, we utilize dimensionality reduction to ensure that 
the number of discrete candidate centers that must be considered is polynomial in both the 
number of points n and in the dimension p. 
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Figure 1: Approximation ratios obtained from opening [3k centers 


Definition 2. In the fc-median problem, we are given a set of points T>, a set of potential 
center locations C and a distance function (d(i, j))iec,jev- The cost of assigning point j to 
center i is d(i,j). Our goal is to open at most k centers and assign each point to a center 
so as to minimize the total cost. 

The first approximation algorithms for the A;-median problem were given by |T6], who gave 
an LP-rounding algorithm that attains an approximation factor of 1 + e by opening 0(klnn) 
centers (i.e. a (1 + e, O(lnn)) bi-criteria approximation). In further work, [15j showed that 
if the distance function d is a metric, it is possible to obtain a 2(1 + e) approximation 
algorithm by opening only (1 + 1 fe)k centers. The first constant-factor approximation for 
the metric k- median problem using only k centers was obtained by [6], who showed that a 
simple local search algorithm gives a 3 + £ approximation. This remained the state of the art, 
until recently, when [IT] gave a 2.732 + e approximation algorithm based on LP rounding. 
Subsequently, this has been improved to 2.611 + £ by [8j. 

Unfortunately, our resulting k- median instance is non-metric, and so we must employ an 
alternative to the standard triangle inequality in our analysis. In the case of our LP-based 
algorithms, we use the fact that our reduction produces instances satisfying a 3-relaxed 3- 
hop triangle inequality, a concept that we define in Section [2] In the case of local search, we 
note that given any partition of points of W into clusters S i,... ,Sk, the optimal location 
of each A;-means cluster Sf s center is the centroid of all points in S). This, combined with 
the fact that our reduction to A;-median approximately preserves the A;-means cluster costs 
allows us to employ a similar approach to that of [ IB] . 
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1.1 Our Results 


We give three approximation algorithms. The first algorithm is based on linear programming. 
It gives 


di (/3) 


1 + e-P 


6/3 

1-/3 P ) 


approximation (see also (fill) for a slightly tighter bound). The second algorithm is based on 
local search. It gives 


M/3) — (1 + 0(e)) ^1 + — j 


approximation. The third algorithm is also based on linear programming. It gives 


a 3 (/3) = max (l + 8e 8 , 


/3(e 1 + 8e /3 ), 

7 =i } 


approximation. The algorithm is similar to the first algorithm, but it uses pipage rounding 
(see HD instead of randomized rounding. In the conference version of the paper, we omit 
the description of the third algorithm. The approximation factors are shown in Figure CD 
In Section [2] we introduce the notation that we shall use throughout the rest of the 
paper and review standard notions related to both the fc-means and /e-median problems. 
In Section [3l we give the details of our reduction to fc-median. Finally, in Sections [4] and 
El respectively, we present our main LP-based algorithm and local search algorithm for the 
resulting /e-median instances. 


2 Preliminaries 

We now fix some notation, and recall some basic properties of /c-means solutions and the 
standard linear program for the fc-median problem. Additionally, we define the notion of an 
ct-rclaxed 3-hop triangle inequality, which will be crucial to the analysis of our LP-rounding 
algorithms. 

2.1 fc-means 

Consider a given instance of the h-means problem, specified by a set of points X e MP. 
Given a partition S = (Si,..., Sk) of X and a set C = (c±, , Ck) of centers in W, denote 
by costx(S', C) the total cost of the clustering that, for each 1 < i < k assigns each point of 
Si to the center cc 

k 

cost x(S, C) = EEii-cii 2 ' 

2—1 x£Si 

Note that to describe an optimal solution to the h-means problem, it is sufficient to specify 
either all clusters or all centers in the solution. Indeed, given a list of clusters S 1; ..., Sk, we 
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can find the optimal assignment of centers q for it: the optimal choice of center q for S) is 
Yl Xj e Si x r For this choice of q, we have 

Sll X_Ci l| 2= 2ik \\ X '- X 'T- (!) 

xGSt * x',x"£Si 

Given a partition S = (Si ,..., Sk) of X into clusters, we then denote by costx('S') the cost 
of this optimal choice of centers. That is, 

cost*(S') = Y Y “ x "^' 

i =1 ' x',x"eSi 

Similarly, given a list C of centers ci,...,q,, we can find the optimal partition S = 
(Si ,..., Sk) of X into clusters. For each c G C, let Nc(c ) be the set of those points x G X that 
are closer to q than to other centers q ^ Ci (if a point x is at the same distance from several 
centers, we break the ties arbitrarily). The optimal partition for C then sets Si = Nc(ci). 
Given a set C of k centers, we define costx(G) = costx(T), where T = (Nc(c\ ),..., N c (ck)) 
is the partition induced by C. 

2.2 ^-median 

We will reduce a given instance X of h-means problem to an instance of the h-median prob¬ 
lem, specihed by (' D,C,d ). By analogy with h-means problem, we can consider a partition 
S = Si ,..., Sk of demand points from V, and then consider the best choice of a single center 
for each partition. We denote the cost of this choice by cost v,d(S): 

k 

cost v, d (S) = E min E d(x,j). 

i= 1 ' j&Si 

Similarly, given a list of k centers C = (q,... ,Ck), let N c (ci) be the set of those demand 
points x G V that are closer (according to the distance function d) to q than to any other 
center in C (again, if a point x is at the same distance from several facilities, we break 
ties arbitrarily). As in the case of h-means, we define costx>,d(C') = cost v,d(T) where T = 
(N c (ci), ..., Nc(ck )) is the partition of V induced by C. 

Although the distance function d in our h-median instances will not satisfy the standard 
triangle inequality, we can show that it satisfies a relaxed variant of the following sort: 

Definition 3. We say that d satisfies an a -relaxed 3-hop triangle inequality on V U C if, 
for any j,f G V and i, i! G C, we have 

d(i,j ) < a (, d(i,j) + d(i',j') + d(i',j)). 

Specifically, we shall show that the distances produced by our reduction satisfy a 3-rclaxed 
3-hop triangle inequality. 
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3 Reduction from /c-means to /c-median 


We now give the details of our reduction from the fc-means to the fc-median problem. In 
the fc-median problem, a hnite set C of candidate centers is specihed, while in the h-means 
problem, the ideal center for each cluster S) of points is given by the centroid of S t . Ideally, 
we want to ensure that for every possible centroid of the original A;-means instance, there is 
some nearby candidate centers in C. The following notion of an e-approximate centroid set, 
introduced by [20], captures this requirement. 

Definition 4. A set of points C C is an e-approximate centroid set for X C if for 
every S C X , 

min > ||x — ell 2 < (1 + e) min > llx — ell 2 . 

esc ^ cgkp 

x£S x&S 

Observe that if C is an e-approximate centroid set for A", then for every set of k centers 
C (in particular, for the optimal set C*), there exists a Appoint subset C C C such that 

costx(C') = min ||x — c|| 2 < (1 + e) min ||x — c|| 2 = (1 + e) cost x(C). 

xex ceC x&x c£C 

Thus, if we restrict our search for k center points in fc-means problem to only those points 
of C, wc lose at most a factor of (1 + e). 

Matousek showed that for every set X in M p and £ > 0, there exists an e-approximate 
centroid set of size 0(|A"|e _p log(l/e)). 

Theorem 5 (Theorem 4.4 in [20]). Given an n-point set X C M p and £ > 0, an e- 
approximate centroid set for X of size 0{ne~ p log(l/e)) can be computed in time 0(nlogn + 
ne~ p log(l/e)). 

Unfortunately, in our setting, the dimension p of the space in which points Xi,... ,x n 
lie may be as large as n. Thus, in order to apply Theorem [0] we first embed X into a 
low-dimensional space using the Johnson-Lindenstrauss transform. 

Theorem 6 (Johnson-Lindenstrauss Flattening Lemma). For every set of points X in 
and e G (0,1), there exists a map tp of X into p = 0(log |A"|/£ 2 ) dimensional space such that 

Ik - 2/111 < Ikk) - <p(y)\\l < (1 + e)lk - y\\l ( 2 ) 

We say that the map ip is a dimension reduction transform for X. 

Given an instance A" of /e-means, we apply the dimension reduction transform to A", get 
a set X' C M p , and then find an ^-approximate centroid set C to X'. We obtain an instance 
(X', C, d) of ^-median with the squared Euclidean distance d. We show in Theorem [7] that the 
value of this instance is within a factor of (1 + e) of the value of instance X of h-means, and, 
moreover, that there is a one-to-one correspondence between solutions of instance (X', C, d) 
and solutions of instance A". We prove Theorem [7] in Section ICl 
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Theorem 7. 1. For every £ G (0,1/2), there exists a polynomial-time reduction from k- 
means to k-median with distance function that satisfies the 3-relaxed 3-hop triangle inequality. 
Specifically, given an instance X of k-means, the reduction outputs an instance (D , C, d) of 
k-median with \D\ = \X\, \C\ = n°^ og< d^ £ ^ e \ and distance d that satisfies the 3-relaxed 3-hop 
triangle inequality such that 


OPT x < OPT (27)d> < (1 + £)OPT x , 

where OPTx is the value of the optimal solution to X and OPT(x>,d> is the value of the optimal 
solution to (' D,C,d ). The reduction also gives a one-to-one correspondence if : V —> X such 
that 

cost x (i>(S)) < cost v, d (S) < (1 + e) cost x (if(S)), 

where S = {Si, ..., Sk) is a partition ofV and if(S) = {if (Si ),..., if(Sk )) is the correspond¬ 
ing partition of X. The reduction runs in time 

2. In instance (V,C,d), C CW 5 (for somep), C is an (e/ 3 )-approximate centroid for V, 
and d(c, x ) = ||c — x|| 2 . 

4 Algorithm for /c-Median with Relaxed Triangle In¬ 
equality 

We now turn to the problem of approximating the ^-median instance from Theorem [0 Our 
first algorithm is based on the following standard linear programming relaxation for the 
/e-median problem: 


min EE z xc d(x, c), (3) 

cGC x£X 

Vc = k , (4) 

esc 

Y.Zx, = 1, VxeX, (5) 

cGC 

Zxc<y c , VceCj'el, (6) 

z X ci y c >0. (7) 


In the integral solution, each variable y c indicates whether the center c is open; and each 
variable z xc indicates whether the point x is assigned to the center c. Constraint (j3J) asserts 
that we should open exactly k centers; constraint (JSJ) ensures that every point is assigned 
to exactly one center; finally, constraint (JHD says that points can be assigned only to open 
centers. In a fractional LP solution, all z xc and y c lie in the interval [0,1]. Note that in the 
integral solution, z xc = y c . if z xc > 0 (as both z xc and y c must be equal to 1). We can slightly 
change any feasible LP solution so it also satisfies this property. Specifically, we split any 
center c which does not satisfy y c = z xc (for some x G X) in two co-located centers Ci and c 2 : 
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one with weight z xc and the other with weight y c — z xc . We distribute the weights z x > c among 
them as follows: we let z x / Cl = min(z x / c , y Cl )] z x > C2 = y C2 — min (z x / c ,y Cl ). Note that this is a 
standard assumption in the A;-median literature. We refer the reader to [22] (see Lemma 1) 
and [10] for more details. The values y c define the measure y on C: y(C ) = X]cecW c - the 
rounding algorithm and in the analysis, it will be convenient to think of this measure as of 
“continuous measure”: That is, if needed we will split the centers into co-located centers to 
ensure that we can find a set of any given measure f.i. 

For every point x G X, let C x = {c G C : z xc > 0}. The set C x contains all centers that 
serve x in the LP solution. Recall that we modify the solution so that y c = z xc if z xc > 0. 
Hence, y c = z xc if x G C x . For every point x G A", we define its LP radius R x as: 

R x = 22 z xc d(x, c) = 22 yc d ( x ’ c )- 

cGC c^lCx 


Observe, that the LP value, which we denote by LP , equals Rx • 

Algorithm. We now describe our LP-rounding algorithm for /c-Medians with relaxed 
3-hop triangle inequality. 


Theorem 8. There exists a ((3, a) bi-criteria approximation algorithm for k-means with 


<*(P) 


1 + e 13 


/_6/3_ 

V1-/3 


+ 


(/3~1)W 
P > 


( 8 ) 


for every (3 > 1. 

The algorithm first solves the LP problem and modifies the LP solution as described 
above if necessary. Then, it partitions all centers into (3k groups Z G Z, each with LP 
measure 1/(3. It picks one center c at random from each group Z with probability (3y c (note, 
that Yh c &z(3yc = !)■ The algorithm outputs the set of (3k chosen centers, and assigns every 
point to the closest center. 

We now describe the construction of Z in more detail. We partition centers into (3k 
groups as follows. For every x G X, we find the unique ball B x around x whose LP weight 
exactly equals 1/(3 (To do so, we may split some centers, and pick some centers in B x at the 
boundary of the ball but not the others). We find a subset of points W such that balls B x 
with x G W are disjoint, and for every point x G X , we also define a “witness” w(x) G W. 
To this end, we sort all points x G X by the LP radius R x in the ascending order, and then 
consider them one by one. For each x G A", if B x is disjoint from all previously chosen balls, 
then we add x to the set W, we set w(x) = x. Otherwise, if B x intersects some other ball 
B x i that is already chosen, we discard B x and set w(x) = x' . If there are several balls B x i 
intersecting B x , we pick the first x' according to our ordering as the witness. Note, that 
Rw(x) A Rx f° r all x. Once, we found a disjoint collection of balls {B x : x G W}, we add 
them to the set Z. We partition centers not covered by U^g y^B x into groups of LP weight 
1/(3 arbitrarily and add these groups to Z. Thus, we obtain a partitioning Z of all centers 
into groups of LP weight 1/(3. 




Analysis. We show that the algorithm returns a valid solution, and then prove an upper 
bound on its expected cost. The algorithm picks exactly one vertex from each group, so it 
always picks f3k vertices. Hence, it always outputs a valid solution. 

We now give an overview of the proof of the upper bound, and then present the details. 
Let S be the set of centers output by the algorithm. Denote the radius of the ball B x by RP. 
For every vertex x, we estimate the expected distance from x to the closest center c in the 
solution S i.e. E[d(x, 5)]. We show that E[d(x, 5)] < a(/3) R x for a(/3) as in equation OS]). 
Since LP = ^P x R x , we conclude that the algorithm has an approximation factor of a(/3). 

Fix x G A". Recall, that C x = {c : z xc > 0} is the set of all centers that serve x in the 
LP solution. We upper bound d(x,S ) by d(x, (C x U B w ^) n S), which is the distance to 
the closest center in C x U B w ^ chosen by the algorithm. Note that the solution S always 
contains at least one center in B w ( x y so {C x U B w ^) D S ^ 0 . For the proof, we pick a 
particular (random) center f(x) G (C x U B w ^) D S. 

We define f(x) using the following randomized procedure. Consider the partitioning Z 
of all centers into groups of measure 1 /(3 used by the algorithm. Let Z = {Z n C x : Z G 
Z\ Z fl C x 7 ^ 0 } be the induced partitioning of the set C x . For all Z G Z we independently 
flip a coin and with probability (1 — e^P y ^) / {(5y{Z)) make the set Z active. We let A C C x 
to be the union of all active sets Z: we say that centers in A are active centers. Let f(x) be 
the center in A fl S closest to x, if A fl 5 ^ 0 ; let f(x) to be the unique center in B w ^ fl S, 
otherwise. We set £ = 0, if A fl S ^ 0 ; and £ = 1, otherwise. Roughly speaking, £ indicates 
whether f(x) G C x or f(x) G B w ( x y. Specifically, it £ = 0, then f(x) G C x \ if £ = 1, then 
f(x) G B w ( x y Note, however, that C x fl B w ^ ^ 0 , and f[x) may belong to C x fl B w ( x) . 

The center f(x) may not be the closest to x, but since f(x) G S, we have 

d(x, S ) < d(x, (C x U B w( . x) ) AS) < d(x, f(x)). 

In Lemma [9] we show that Pr(£’ = 0) = 1 — eThus, 

E[d(x,f(x)] = Pr(£ = 0) E [d(x, f(x)) | £ = 0] + Pr(£ = 1) E [d(x, f(x)) \ £ = l] 

= (1 - e~ p ) E [d(x, f(x)) | £ = 0] + e-P E [d{x, f(x)) \£ = l]. 

We bound the expected distance from x to f(x) given £ = 0 in Lemma (TUJ We show that 

E [d(x, f(x)) | £ = 0] < R x . (9) 

Observe that for a random center c distributed according to the LP measure in C x (i.e., 
Pr(c = Co) = y(co)/y(C x ) = y(co)), we have the exact equality E[d(x, c)] = R x . So Lemma UOl 
shows that the distribution of f(x) given £ = 0 is “not worse” than the distribution according 
to y in C x . We now proceed to bound the expected distance from x to f(x) given £ — 1. 
Recall, that w(x) is the witness for x. Thus, the balls B x and B w ^ intersect and R. w ( x ) < R x . 
Let c 0 be an arbitrary center in B x D B w m. By the relaxed 3-hop triangle inequality, 

d(x, f(x)) < 3 (d(x, c 0 ) + d(w(x ), c D ) + d(w(x ), f(x)) 

< 3 (RP + R^ x) + d(w(x),f(x))). 
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Here, we used that R% is the radius of B x ; R^ w ^ is the radius of B w ( x y By the Markov 
inequality, Rf. < f3R x /(f3 — 1) (see Lemma fl5l) . In Lemma [T4], we show that there exists two 
nonnegative numbers rq and r 2 (ri < r 2 ) such that 


R i{x) + E[d(u;(x), f(x)) \ f(x) G B w(x) \ D x , S = 1] < rq + r 2 , 


and 


1-7 


r i + 


/ 3~1 

0 


Tg < i?x, 


where 7 is some parameter in [0,1]. Hence, 


E[d(x, f(x)) | f(x) G £„,(*) \ L> x ; £ = 1] < 3 (|—y + r i + Bj) 

By Lemma fill 

Pr(/(x) G £„,(*) \ D x | £ = 1) = e 7 (l - 7 ). 

Finally, in Lemma fT3l we show that 

PRx 


E[d(x, f(x)) | f(x) G D x , £ = 1] < 


Combining all bounds above we get the following inequality: 
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( 10 ) 


E[d(xJ(x))} < ( (1 — e p )+e ^(e 7 (l - 7 ) x 3 ( + ri + ^ + (1 - e 7 (l - 7 )) x 


/9 — 1 


We now find the maximum of the right hand side over all possible values of 7 G [0,1] 
and 77 ,r 2 > 0 satisfying linear inequalities rq < r 2 and (ITU . The right hand side is a linear 
function of rq and r 2 . Hence, for a fixed 7 the maximum is attained at one of the two extreme 
points: (rq,r 2 ) = (0, f3R x /{(3 - 1)) or (77, r 2 ) = {/3R x /(l3 - 7 ), /3R x /(/3 - 7 )). Substituting 77 
and r 2 in the previous inequality we get the following bound on the ratio E [d(x, f(x))]/R x : 


max 

7 e[o 


% (d - + 3e-<^)(l - 7 ) (^ + - (^, ^-)) + 


f3 2/3 


/3e ^(1 — e 7 (l — 7 )) \ 
7 ' 


( 11 ) 

This function can be upper bounded by at({3) defined in (jSJ). We conclude that the approxi¬ 
mation factor of the algorithm is upper bounded by a(/3). 


4.1 Detailed Analysis of the LP Rounding Algorithm 

Lemma 9. We have Pr(£ = 0) = 1 — e -/3 . 

Proof. Recall, that the algorithm picks one center c in every Z G Z uniformly (with respect 
to the measure y) at random. Thus, the probability that the algorithm picks a center from 
Z equals fdy(Z). The probability that a given Z contains a point from the solution S and Z 
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is active equals / 3y(Z) x (1 — e^^) / (f3y(Z)) = (1 — e^^). The probability that no such Z 
exists equals 


n 


e -py(z) = e -E z e zM z ) = e ~y{Cx) = e ~P 


ZGZ 


□ 


We now bound E[d(x, f(x)) \ S = 0]. 

Lemma 10. We have E[d(x, f(x)) \ £ — 0] < R x . 


Proof. We define two sets of random variables P and Q, and then show that they are 
identically distributed. If the algorithm picks a center c in Z , and Z is active, let P(Z) = c. 
Let P(Z) =_L, otherwise. The random variables P(Z) are mutually independent for all 
Z G Z\ and 


Pr (Z = c ) 


(1 _ e ~MZ))y c 

y(z.) 


for c G Z. 

To define Q , we introduce an auxiliary Poisson arrival process. At every point of time 
t G [0,/?], we pick a center c G C x with probability y c dt (i.e., with arrival rate y c ). For every 
Z, let Q(Z) be the first center chosen in Z. If no centers in Z are chosen, we let Q(Z) =_L. 
Note that we pick two centers at exactly the same time with probability 0, hence Q(Z) is well 
defined. Conditional on Q(Z) ^_L, the random variable Q(Z) is uniformly distributed in Z 
with respect to LP weights y c (since at every given time t, the probability of arrival equals 
y c dt). Then, Pr (Q(Z) ^±) = (1 - e~^). Hence, Pr (Q(Z) = c) = (1 - e~ Pv ^)y c /y(Z). 
Note that all random variables Q are mutually independent. Thus, the random variables Q 
have the same distribution as random variables P. 

Note that if £ — 0, then f(x) is the closest center in {P(Z) : Z G Z; Z to x. If 
S — 1, then all P(Z) are equal to _L. Let Uq = {Q(Z) : Z G Z;Q ^T}. Since P and Q 
have the same distribution, we have 


E[d(x, f(x)) | £ — 0] = E[min d(x, c) \ U q ^ 0\. 

c&Uq 


Conditional on U q 0 , the first center that arrives according to our stochastic process is 
uniformly distributed in C x . The expected distance from it to x equals R x . This center 
belongs to U c . Hence, E[min cer / Q d(x, c) \ U q ^ 0\ < R x . □ 

Let D x = B w (x) D C x and 7 = f3y(D x ). Note that 7 G [0,1], since y(B w ( x f) = 1/fd. We 
find Pr(/(x) G D x \ S — 1). 

Lemma 11. We have 


Pr(f(x)eD x \£ = l) = l-e\l- 1 ). 
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Proof. Observe that the set D x = B W ^C\C X is one of the sets in the partitioning Z as w(x) G 
W and B w ^ G Z. Assume f{x) G D x and £ = 1. Since f(x) G D x , we have S' fl D x 0. 
Thus, D x must be inactive (otherwise, £ would be 0). Moreover, for every Z D x (Z G Z), 
Z is inactive or Z 0 S = 0 (again, otherwise, £ would be 0). Hence, the event {f(x) G 
D x and £ = 1} can be represented as the intersection of the following three independent 
events: {S 0 D x ^ 0}, {D x is not active}, and {there are no active vertices in (C x \ D x ) 0 
S}. The probability of the first event is / 3y(D x ); the probability of the second event is 
1 — (1 — e~ i5y ( D 0)/Q 3y(D x )); the probability of the third event is e~^ Cx ^ Dx ^ (this probability 
is computed as in Lemma [HD . Thus, 


/ i _ P -Py( D A \ 

Pr {f{x) G D x and £ = 1) = (3y{D x ) x (l - ) x e ~^ c ^ 

= (7 — (1 — e -7 )) x eU/ 3 - 7 ) = e /3 (l — (1 — 7 )e 7 ). 

This finishes the proof. □ 

We bound E[d(x, f(x)) \ f(x) G D x ; £ = 1] and E [d(x, f(x)) \ f(x) G B w{x) \ D x \ £ = 1] 
in Appendix [A] 

5 Local Search 

For smaller values of (3, we consider the standard local search algorithm (see, e.g., P) for 
the /9/c-median problem using swaps of size p. The algorithm works as follows: we maintain 
a current solution A comprising /3k centers in C. We repeatedly attempt to reduce the cost 
of the current solution A by closing a set of at most p centers in A and opening the same 
number of new centers from C \ A. When no such local swap improves the cost of the 
solution A we terminate and return A. In order to simplify our analysis, we do not worry 
about convergence time of the algorithm here. We note that by applying standard techniques 
(see Pi), we can ensure that, for any 6 > 0, the algorithm converges in time polynomial 
in n — |C U T>\ and | by instead stopping when no local swap improves the cost of A by a 

factor of ^1 — pol y( n ) ^; the resulting algorithm’s approximation ratio increases by only 

Unfortunately the analyses of Pi relies heavily on the triangle inequality, while the 
instances generated by Theorem |T] satisfy only a 3-relaxed 3-hop triangle inequality. Thus, 
we proceed as in |I3] . 

Let O — (o\,... ,Ok) be an optimal set of k centers, and A = (a\,..., apk) be the set of 
/ 3k centers produced by the local search algorithm. As in ra. we say that a center a G A 
captures a center o G O if a is the center of A that is closest to o. Note that each center in A 
can potentially capture several centers in O, but each center in O is captured by exactly one 
center of A. We now construct a set of local swaps to consider in our analysis. We say that 
a center in A is “good” if it does not capture any center of O. Then, because each center of 
O is captured by only one center of A, we must have at least (3k — k = ((3 — l)k good centers 
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in A. We fix some such set of (/? — 1 )k good centers; we call them “auxiliary” centers and 
set them aside for now. 

For the remaining k centers B C. A, we proceed exactly as in [13]: we assign each center 
in O to the bad center of B that captures it. This creates a partition Oi ,..., O r of centers in 
O. We similarly partition the centers of B into r parts Bi ,..., B r with \B{\ = |Oj|; for each 
1 < i < r, let Bi contain the bad center of B that captures all of O* together with \Bi\ — 1 
unique good centers of B. Note that the fact that each center of O is captured only once 
ensures that there are indeed enough good centers in B for our construction. Now, we use 
this partition of B and O to construct a set of swaps, each assigned some weight. If |Oj| < p, 
we consider the (. B^Oi ) with weight 1. If 10*| = q > p, we consider a group of singleton 
swaps ({&}, jo}), where o G O* and b is a good center in B i: each with weight A-p At this 
point, note that every center in O occurs in swaps of total weight 1, and every center in B 
occurs in swaps of total weight at most < 1 + A Now, we add swaps involving auxiliary 
centers; for each of the (/3 — 1 )k auxiliary centers a G A \ B and each o E O, we consider 
singleton swap ({a}, {o}), assigned weight -k Each center of O now occurs in swaps of total 
weight 1 + (/? — 1) = /3, while each center of A \ B occurs in swaps of total weight 1. 

Summarizing, our set of swaps satisfies the following properties: (1) each center of O 
occurs in swaps of total weight /3; (2) each center of A occurs in swaps of total weight at 
most 1 + -; (3) for any swap (A', O') in our set, no center in A 1 captures any center not in 
O'. We now give a brief sketch of how these properties lead to our desired approximation 
ratio (we give a full description of the analysis in the appendix). Our analysis closely follows 
that of [13]. 

As in ra, the total change cost v,d(A \ A' U O') — costx> ) rf(A) due to performing a single 
swap (A', O') is at most: 

E E (d(x,o) - d(x,a x )) + E E (d{x,a 0 J -d(x : a x )). 

oGO' xGN 0 (o) aeA' xGN a (A') 

If A is locally optimal, then we must have that cost^^A \ A' U O') — costx>,d(A) > 0 for all 
swaps (A', O') considered by the algorithm. In particular, for each swap (A', O') in our set, 
we have: 


osE E (d(x, o ) — d(x, a x )) + E E (d(x,a 0x ) ™ d(x,a x )). (12) 

oEO' xGNq(o) clGA' x£Na(A') 


Multiplying each inequality (fl2l) by the weight of its swap and then adding the resulting 
inequalities we obtain: 


0 < A TXd(x,o x ) - d(x,a x )) + f 1 + - j y^(rf(s,a 0 J - d(x,a x )), 
xev k V J x&v 


due to properties (1) and (2) of our set of swaps. Theorem [7] part 2, which shows that our 
center set is an approximate fc-means centroid set, then allows us to simplify the final term 
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above as in [13], giving: 


0 < (p + 2 + |) cost-D jd (0) - -costx>,d(^4) + 0(e) • cost v,d.{A), 

where a 2 = costp’d(O) * s the sc L uare d approximation ratio of our algorithm. Rearranging and 
simplifying (again, we give a detailed analysis in the appendix), we obtain. 

A 2 2 \ 1 

a< \ + 0 + Pp) 1 — 0(e) 

Therefore, we have proved the following theorem: 

Theorem 12. There exists an algorithm that produces a solution for any instance of /3k- 
median problem satisfying the properties of Theorem [?} where (3 > 1 is a fixed constant. For 
any p > 1 and any e £ (0,1] , the algorithm runs in time polynomial in \C U T>\ and produces 
a solution A satisfying: 


costv,d(A) <(l + | + £) -MO) 

where O is the optimal set of k centers in C. 
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A Detailed Analysis of the LP Rounding Algorithm 


In this section, we bound E [d(x,f(x)) \ f(x) G D x \£ = 1] and E [d{x,f(x)) \ f(x) G 
B w{x) \D x -£ = 1], 

Lemma 13. The following bound holds: 

E[d(x,f(x))\f(x)eD x ,£ = l}<^. 

7 

Proof. Given f(x) G D x and £ — 1, the random center f(x) is distributed uniformly in D x 
(with respect to the LP weights y). Hence, Pr (f(x) = c) = y c /y(D x ) for c G D x . We have 


E[d(x, f(x)) | f(x) G D x ; £ = 1] 


E ceDx Vcdjx, c ) < Ecgc, Vcdjx, c ) 

y(D x ) ~ y{D x ) 


R 


X 


7 /P' 


□ 


Lemma 14. There exists two nonnegative numbers r i and r 2 satisfying 

1. 


1~7 


n + 


/3-l 


2 . 


r\ < r 2 , 


T 2 < Rx, 


such that 

R 'l(x) + Hd(w(x), f(x)) | f(x) G B w{x) \ D x ; £ = 1] < n + r 2 . 

Proof. Denote the expected distance from a random center c in B w ( x ) \ D X to w(x) by r\ and 
distance from a random center c in C w ( x ) \ B w ^ to w(x) by r 2 : 


r i = 


E 


y c d(w(x), c); r 2 = 


E 


y c d(w(x ) 


c . 


c €Bw(x)\D x 


°€Cw(x) w(x ) 
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By the definition of R w ( x \ , we have 


R 


w(x) 


( ^2 Vcd{w(x), c) j + y(B w ( x ) \ D x ) \ P*) r 2 

c(ElD x 


> 



fi-1 

/3 




Note that -R u ,( x .) < R x . Hence, 


1~7 

0 


ri + 


l-l 

/? 


r'2 < P* 


Since all centers in \ D x lie inside of the ball of radius R^x) around w(x), and all 

centers in C w ^ \ B w ^ lie outside of this ball, we have rq < R'^x) A T i- Hence, R w ( x ) + 
d(w(x), f(x)) < r 2 + d(w(x), f(x)) and rq < r 2 . Conditional on f(x) G B w ( x ) \ D X and £ — 1, 
the random center f(x) is distributed uniformly in B w ( x ) \ D x with respect to the weights y. 
Hence, E[d(tc(a;), f(x)) \ f(x) G B w ^ \ D x ; £ = 1] = ri. Consequently, 

P^*) + E[d(iu(x), f(x)) | /(x) G B w{x ) \ D x , £ = 1] < n + r 2 . 


□ 


Lemma 15. The following inequality holds: R x < I3R X /(I3 — 1). 


Proof. We have 


P, 


^2y c d(x,c)< ^2 Vcd{x,c). 

c€C x c&C x \B x 


Every center c G C x \ B x is at distance at least Pf from x. Hence, 


R,< E VcRl = v(C,\B,)Ri = (i-\)Ri 

ceC x \B x 


The desired inequality follows. 


□ 


B Detailed Analysis of the Local Search Algorithm 

Here we give a detailed analysis of the local search algorithm from section [5] closely following 

ra- 

For a set of points P C X and a point c G M p , define the total distortion of P with 
respect to c as A(P, c) = J2 P eP II P ~ c l| 2 - We shall use the following Lemmas from [13]: 

Lemma 16 (Lemma 2.1 in [13]). Given a finite subset P of points in MP, let c be the centroid 
of P. Then, for any c' G M p , A (P, c') = A (P, c) + |P| • ||c — c'|| 2 
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Lemma 17 . Let (pf) and (£*) be two sequences of reals such that a 2 = {Yli Pi) / (Yli £i) f or 
some a > 0 . Then, 

n 1 n 

X>&<- 

i=1 i= 1 

We now show how local optimality implies the desired inequality. For a demand point 
x G V, let a x and o x denote the closest facility to x in A and O, respectively. Recall that 
for for a G A, N^(a) is precisely the set of all those demand points x G V such that a x = a, 
and, similarly, for o G O, No(o ) is the set of all demand point x G D such that o x = o. Now, 
we upper bound the change in cost clue to some swap (A', O') in our set of swaps. We do 
this by constructing a feasible assignment of all points in T> to centers in j4\LU O'. For 
each o G O' , we assign all the points in N 0 {o) to o. This changes the cost by 

E E (dfx , o) — dfx, a x )). 

oGO' xGNo(o) 

Now, fix a point x G Na(A') \ N 0 ( 0 '), and consider ads closest optimal center o x . We must 
have o x qL O’. Let a 0x be the closest center to o x in A. Then, by property (3) above, a 0x ^ A', 
since a Qx captures o x but o x ^ O'. We reassign x to a Qx . The total cost of reassigning all 
such points x is at most: 

E E (d(x,a 0x ) - d(x,a x )) < E E (d(x,a 0 J - d(x,a x )), 

aeA' x£N a (A')\N 0 { 0 ') a&A’ x£N a (A') 

where the inequality follows from the fact that a x is the closest center to x in A, and so 
d(x, a Qx ) — d(x , a x ) > 0 for all x G Na(A') D N 0 { 0 '). Thus, the total change costx>^(A \ A' U 
O') — costx>,d(dL) for each swap (W, O') is at most: 

E E (d(x, o) — dfx, a x ) + E E (d(x,a 0 J - d(x,a x )). 

oGO' x^No(o) clEA' x€.Na(A') 

If A is locally optimal, then we must have that costx> )( ;(dL \ A' U O') — costx> i( 2 (W) > 0 for 
all swaps {A ', O') considered by the algorithm. In particular, for each swap {A ', O') in our 
set, we have: 

0 < E E (d(x, o) — d(x, a x )) + E E (d(x,a 0x ) - d(x,a x )). 

oGO' x£N 0 (o) aeA’ xGNa(A') 

Set 7 = 1 + K Then, multiplying each such inequality by the weight of its swap and then 
adding the resulting inequalities we obtain 

0 < P ^ (d(x, o x ) - d(x, a x )) + 7 ^ a °*) ~ 

x x€fD 

= /3 cost v,d(0) - (ft + 7) cost v,d{A) + 7 ( 13 ) 

x£T> 
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where we have exploited properties (1) and (2) of our set of swaps to bound the number of 
times a given center in 0 or A is counted in our sum of inequalities. 

It remains to bound the final term in (I13j) . Consider some o G O, and let c be the centroid 
of No(o). As above, we will let a Q denote the closest center in A to O. Then, note that: 

A (N 0 (o),a 0 ) = A(N 0 (o), c) + |IV 0 (o)| • ||c - a D || 2 (Lemma [16]) 

< A(A r 0 (o), 6) + |IVo(o)| • ||c — a 0 || 2 (c is the centroid of N 0 {o)) 

< 22 [ d ( x , °) + (1 + e)||o- a 0 || 2 ] 

x£No(o) 

(Theorem [7] part 2, and the fact that o is optimal center for Nq(o)) 

< 22 \d(x, o) + (1 + e)||o — a x || 2 ] . (a G is the closest center to o in A) 

x£N 0 (o) 

<(l + e) 22 [d( x ,o) + ||o — a x || 2 ] . 

x&Nq(o ) 


Let a 2 = c ° h ' D ' d ln\ — tT' eP be the approximation ratio attained by the algorithm. 

COStx),d(,L^J / .ygf) dt(3?,Oa;) 

Summing over all o G O, and recalling that for all x G Nq(o ) we have o x = o, we obtain: 


22 a °^ = 22 A ( iV o(°)’ a °) 

x€.T> oGO 


< (1 + e) 22 22 [d(x,o) + \\o - a x \\ 2 ] 

oGO xGNq(o) 

= (1 + e) 22 i d ( x ’ ° x ) + II 0 * _ a x\\ 2 ] 

x£V 

= (1 + e) [d(x, o x ) + ||x - o x || 2 + \\x - a x \\ 2 + 2 \\x - o x \\||x - a x ||] 

xGV 


= (1 + e) 22 [2d(x, o x ) + d(x,a x ) + 2\\x - o x \\\\x - a x \\] 
xev 

<(l+e)Y. 

xGT> 


2d(x, o x ) + d(x, a x ) H— d(x, a x ) 

a 


— (1 + £ ) 


2 cost v,d(0) + 



cost v,d( A ) 


(14) 


Where in the last inequality, we have applied Lemma [T7] to the sequences pi and Ci defined 
by: 

2 _ Y2xev d(x, a x ) _ 22=1 P i 
YjxZD d (x, o x ) Ytiti' 

Applying the upper bound (fT4l) to the final term of (fT3]l . we obtain: 


0 < f3 cost v, d {0) - (/3 + 7 ) cost v, d ( A ) + 7(1 + £ ) [ 2 costx, jd (0) + (l + f) costx> id (A)] 
<{/3 + 2y) cost v,d(0) - (ft - costi ^d(A) + (3 + |) je costx>,d(A) 
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where we have used the fact that costp^O) < cost©, ^ (A). Rearranging, we have 
(P + 27 ) costx>,d(0) > (p - ^ - (3 + f) 7 e^j cost v,d( A ) 


which implies: 


Thus, we have: 


= (p ~ ^ - (3 + l ) 7^ « 2 costD, d (0), 


a 2 p — 27a — P — 27 — a 2 (3 + 76 < 0 

2 27a 27 a 2 /q 2\ / n 

“ - — - 1 -j-y( 3 + d^<o 

27 \ a 2 


(a + 1) ( a - 1 - — J - — (3 + l ) 7 e < 0 


a — 1 — 


27 

P 


a 


(oc + 1 )P 


(3+I)t£<0. 


27 2 2 

(1 - 0( e )) a <n-_ = i + _ + _. 


C Proof of Theorem \ 7 \ 

In this section, we prove Theorem [71 Consider an instance of fc-means with a set of points 
X C Denote n = |W|. Let s' = e/3. Let ip : M p —» M? be a dimension reduction transform 
for X with distortion (1 + s') as in Theorem [ 6 j Note that p = 0(\ogn/e' 2 ) = 0(\ogn/e 2 ). 

Let X' = p>(X) C ML Using the algorithm from Theorem [5], we compute an e'- 
approximate centroid set C cP for X'. The size of C is 

0 (n£~P log(l/e:)) = 7 i£~° ( ' logn / £2) log(l/e:) = n ■ ^(^sU/D/e 2 ) — n °(i°gdAd/e 2 ). 

we need time 0 (nlogn + ne~^ log(l/e)) = to compute it. 

We first show that for every solution of the fc-means problem on X there is a corresponding 
solution of /c-means problem on X' in which all centers lie in C, and vice versa. 

Lemma 18. 1. For every partition S = (Si ,..., Sk) of X, there is a corresponding clus¬ 

tering of X' given by S' = (<p(Si ),..., </?(£*)) and some centers C = (c \,..., c' k ) C C 
such that: 

cost X '(S',C') < (1 + e') 2 cost x (S). 

2. For every partition S' = (S [,..., S' k ) of X', there is a corresponding clustering S = 
(</j~ 1 (S'i),... , 9 ? _ 1 (S'/ C )) of X and some centers C = (< 7 ,... ,< 7 ) C W such that 

cost x(S,C) < costx'(S'). 
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Proof. Part HJ Consider a partition S = (Si ,..., Sk) of X and the corresponding partition 
S' = (S [,..., S' k ) of X', where S( = ip (Si). Let c- = argmin ceC YlxeS'. \\ x ' ~ c ll 2 f° r ' l e 
{1,..., k}. Because C is an s '-approximate centroid set for X', we have, for each cluster S', 


Y ||x — c'|| 2 < (1 + e') min 


x — c = 


xes' 


ce s' 


E ii 1 ' 


„// II 2 


X 


x' ,x"&S[ 
_/\2 


1+ ( Y, lkM-^")ll 2 <^L E II 1 ' 


2\S'\ 

1 x',x"£Si 


X 


//|| 2 


x',x"GSi 


Hence, 


k 

cost X' (S') = EE ||x — c'|| 2 < (1 + e') 2 costx(S). 


i =i xeSi 

Part [21 Consider a partition S' = (S [,..., S' k ) of X' and the corresponding partition 
S = (Si ,..., Sk) of A", where Si = ip^ 1 (S i ). Define the centers q = Y2xeSi x /\^i\- Then, for 
each cluster Si, we have: 



xGSi 


1 y \\x'—x "\\ 2 < —r^-j- y n^(^o—^(^")ii 2 


1 x',x"es t 


2\SA < 

1 11 x',x"&Si 


v\ E ii x ' 


// 112 


2|S' , | ' , 

1 11 x',x"es 


x 


Hence, 


fc 

cost A' (S) = EE ||x — Cj|| 2 < COStx'(*S"). 

i =1 x£Si 


□ 


Now we are ready to define instance (V,C,d). Let V = X' , C be the ^-approximate 
centroid we defined above, and d(c,x) = ||c — x|| 2 for every c e C and x e V. Define 
ip : T2 — y X by ip(x) = ip~ l (x). 

We prove that our reduction, which maps instance A" of /c-means to instance (T>, C, d) of 
^-median, satisfies the conditions of the theorem. 

Lemma 19. Our reduction produces an instance that satisfies the following properties: 

1. The distance function d satisfies the 3-relaxed 3-hop triangle inequality onVUC. 

2. For every partition S = (Si,...,Sk) of V and the corresponding partition ip(S) = 
(ip(Si),..., i>(Sk)) of X, we have 


cost x (i>(S)) < cost v,d(S) < (1 + e) costx(ip(S)). 
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3. We have 

OPT* < OPT ( x> )d> < (1 + £)OPT*. 

Proof. Claim Q] follows from the fact that: 

||x — vj \\ 2 < (||x — y\\ + \\y — z\\ + \\z — iu||) 2 < 3(||x — 2/11 2 + \\v ~ z \\ 2 + \\ z ~ w|| 2 )- 
for any w, x,y,z G W. 

For claim [21 consider any partition S of T>. Let T = be the corresponding partition 
of X, given by T* = if(Si). Then, from our definition of d, we have cost v,d(S) = cost*/(5). 
Moreover, by Lemma [T8l we have cost*/ (S) is between cost*(T) and (1 + e') 2 cost*(T). 
Thus, 


cost*('0(5)) < cost v,d(S) < (1 + e ') 2 cost*(V>(S')) < (1 + e) cost* (i>(S)). 

Since for every partition S of C there is a corresponding partition if(S) of A", and for every 
partition T of X there is a corresponding partition <p(T) of T>, we immediately get from 
claim [2] that OPT* < OPTp^ < (1 +s)OPT*. 

□ 
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